Neuromorphic vibrotactile stimulation of fingertips for encoding object stiffness in telepresence sensory substitution and augmentation applications

We present a tactile telepresence system for real-time transmission of information about object stiffness to the human fingertips. Experimental tests were performed across two laboratories (Italy and Ireland). In the Italian laboratory, a mechatronic sensing platform indented different rubber samples. Information about rubber stiffness was converted into on-off events using a neuronal spiking model and sent to a vibrotactile glove in the Irish laboratory. Participants discriminated the variation of the stiffness of stimuli according to a two-alternative forced choice protocol. Stiffness discrimination was based on the variation of the temporal pattern of spikes generated during the indentation of the rubber samples. The results suggest that vibrotactile stimulation can effectively simulate surface stiffness when using neuronal spiking models to trigger vibrations in the haptic interface. Specifically, fractional variations of stiffness down to 0.67 were significantly discriminated with the developed neuromorphic haptic interface. This is a performance comparable, though slightly worse, to the threshold obtained in a benchmark experiment evaluating the same set of stimuli naturally with the own hand. Our paper presents a bioinspired method for delivering sensory feedback about object properties to human skin based on contingency-mimetic neuronal models, and can be useful for the design of high performance haptic devices

Functional mimicry of Ruffini receptors with fibre Bragg gratings and deep neural networks enables a bio-inspired large-area tactile-sensitive skin

Collaborative robots are expected to physically interact with humans in daily living and the workplace, including industrial and healthcare settings. A key related enabling technology is tactile sensing, which currently requires addressing the outstanding scientific challenge to simultaneously detect contact location and intensity by means of soft conformable artificial skins adapting over large areas to the complex curved geometries of robot embodiments. In this work, the development of a large-area sensitive soft skin with a curved geometry is presented, allowing for robot total-body coverage through modular patches. The biomimetic skin consists of a soft polymeric matrix, resembling a human forearm, embedded with photonic fibre Bragg grating transducers, which partially mimics Ruffini mechanoreceptor functionality with diffuse, overlapping receptive fields. A convolutional neural network deep learning algorithm and a multigrid neuron integration process were implemented to decode the fibre Bragg grating sensor outputs for inference of contact force magnitude and localization through the skin surface. Results of 35 mN (interquartile range 56 mN) and 3.2 mm (interquartile range 2.3 mm) median errors were achieved for force and localization predictions, respectively. Demonstrations with an anthropomorphic arm pave the way towards artificial intelligence based integrated skins enabling safe human–robot cooperation via machine intelligence

Cardio-Respiratory Monitoring in Archery Using a Smart Textile Based on Flexible Fiber Bragg Grating Sensors

In precision sports, the control of breathing and heart rate is crucial to help the body to remain stable in the shooting position. To improve stability, archers try to adopt similar breathing patterns and to have a low heartbeat during each shot. We proposed an easy-to-use and unobtrusive smart textile (ST) which is able to detect chest wall excursions due to breathing and heart beating. The sensing part is based on two FBGs housed into a soft polymer matrix to optimize the adherence to the chest wall and the system robustness. The ST was assessed on volunteers to figure out its performance in the estimation of respiratory frequency (fR) and heart rate (HR). Then, the system was tested on two archers during four shooting sessions. This is the first study to monitor cardio-respiratory activity on archers during shooting. The good performance of the ST is supported by the low mean absolute percentage error for fR and HR estimation (≤1.97% and ≤5.74%, respectively), calculated with respect to reference signals (flow sensor for fR, photopletismography sensor for HR). Moreover, results showed the capability of the ST to estimate fR and HR during different phases of shooting action. The promising results motivate future investigations to speculate about the influence of fR and HR on archers’ performance

Assessment of intuitiveness and comfort of wearable haptic feedback strategies for assisting level and stair walking

Nowadays, lower-limb prostheses are reaching real-world usability especially on ground-level walking. However, some key tasks such as stair walking are still quite demanding. Providing haptic feedback about the foot placement on the steps might reduce the cognitive load of the task, compensating for increased dependency on vision and lessen the risk of falling. Experiments on intact subjects can be useful to define the feedback strategies prior to clinical trials, but effective methods to assess the efficacy of the strategies are few and usually rely on the emulation of the disability condition. The present study reports on the design and testing of a wearable haptic feedback system in a protocol involving intact subjects to assess candidate strategies to be adopted in clinical trials. The system integrated a sensorized insole wirelessly connected to a textile waist belt equipped with three vibrating motors. Three stimulation strategies for mapping the insole pressure data to vibrotactile feedback were implemented and compared in terms of intuitiveness and comfort perceived during level and stair walking. The strategies were ranked using a relative rating approach, which highlighted the differences between them and suggested guidelines for their improvement. The feedback evaluation procedure proposed could facilitate the selection and improvement of haptic feedback strategies prior to clinical testing

Haptic Glove and Platform with Gestural Control For Neuromorphic Tactile Sensory Feedback In Medical Telepresence ^†

Advancements in the study of the human sense of touch are fueling the field of haptics. This is paving the way for augmenting sensory perception during object palpation in tele-surgery and reproducing the sensed information through tactile feedback. Here, we present a novel tele-palpation apparatus that enables the user to detect nodules with various distinct stiffness buried in an ad-hoc polymeric phantom. The contact force measured by the platform was encoded using a neuromorphic model and reproduced on the index fingertip of a remote user through a haptic glove embedding a piezoelectric disk. We assessed the effectiveness of this feedback in allowing nodule identification under two experimental conditions of real-time telepresence: In Line of Sight (ILS), where the platform was placed in the visible range of a user; and the more demanding Not In Line of Sight (NILS), with the platform and the user being 50 km apart. We found that the entailed percentage of identification was higher for stiffer inclusions with respect to the softer ones (average of 74% within the duration of the task), in both telepresence conditions evaluated. These promising results call for further exploration of tactile augmentation technology for telepresence in medical interventions

Uneven Terrain Recognition Using Neuromorphic Haptic Feedback

Recent years have witnessed relevant advancements in the quality of life of persons with lower limb amputations thanks to the technological developments in prosthetics. However, prostheses that provide information about the foot–ground interaction, and in particular about terrain irregularities, are still missing on the market. The lack of tactile feedback from the foot sole might lead subjects to step on uneven terrains, causing an increase in the risk of falling. To address this issue, a biomimetic vibrotactile feedback system that conveys information about gait and terrain features sensed by a dedicated insole has been assessed with intact subjects. After having shortly experienced both even and uneven terrains, the recruited subjects discriminated them with an accuracy of 87.5%, solely relying on the replay of the vibrotactile feedback. With the objective of exploring the human decoding mechanism of the feedback startegy, a KNN classifier was trained to recognize the uneven terrains. The outcome suggested that the subjects achieved such performance with a temporal dynamics of 45 ms. This work is a leap forward to assist lower-limb amputees to appreciate the floor conditions while walking, adapt their gait and promote a more confident use of their artificial limb